1. Field of the Invention
The invention relates to encoders and image encoding methods, and more particularly to encoders and image encoding methods used in seamless bitstream switching.
2. Description of the Related Art
In conventional commercial video streaming, the server may store multiple bit-streams with different bitrates/resolutions to deal with bandwidth variation in standard compliance beforehand. With bitstream switching, the server chooses the bitstream which matches the client's available bandwidth. For example, clients with high channel bandwidth can subscribe to higher bitrate bitstreams for better video quality, whereas clients with low channel bandwidth subscribe to lower bitrate bitstreams with lower video quality.
FIG. 1A and FIG. 1B show conventional bitstream switching with direct switching. In FIG. 1A, as the client's available bandwidth decreases, bitstreams transmitted by server are switched from high bitrate bitstreams (H) to low bitrate bitstreams (L). The dotted blocks PH1, PH2, PL3, PL4, and PL5 in FIG. 1A are frames received by the client. In FIG. 1B, as the client's available bandwidth increases, bitstreams transmitted by server switch from low bitrate bitstreams (L) to high bitrate bitstreams (H). The dotted blocks PL1, PL2, PH3, PH4, and PH5 in FIG. 1B are the frames received by the client.
As shown in FIG. 1A, when bitstreams are switched from high bitrate bitstreams (H) to low bitrate bitstreams (L) at time t, the frames received by the client are the dotted blocks PH1, PH2, PL3, PL4, and PL5 in sequence. In the encoder, predicted frame PL3 is encoded according to frame PL2. In the decoder, frame PL3 is decoded according to the decoded frame PH2. Because of the different reference frames in encoder and decoder, drift occurs at frame PL3.
To avoid drift from bitrate switching, standard H.264 provides seamless bitstream switching. Standard H.264 defines a novel frame type, SP frame. Note that SP frame and P frame are all generated by predicted decoding according to time.
FIG. 2A shows a conventional server switching low bitrate bitstreams (L) to high bitrate bitstreams (H) by SP frame when the client's available bandwidth increases. The frames received by the client are the dotted blocks PL1, PL2, SPLH, PH4, and PH5 in sequence. Note that the frame at switching point t is encoded to SP frame. SP frame comprises primary SP frames SPH and SPL, and a secondary SP frame SPLH. Thus, when the bitstream switches, drift errors are avoided by transmitting bridge SP frame SPLH to client at switching point t.
SP frames (SPL, SPH, and SPLH) are obtained by encoder 300 in FIG. 3 according to P frame PH3 and the original P frame PL3 at switching point shown in FIG. 1A.
Encoder 300 comprises low bitrate encoding unit 310, high bitrate encoding unit 350, and bridge frame encoding unit 340, and generates SP frames SPH, SPL, and SPLH according to P frame PH3 and PL3.
Low bitrate encoding unit 310 comprises transformation unit 312, adder 314, quantization unit 316, statistic encoding unit 318 and feedback circuit 320. Transformation unit 312 receives P frame PL3 and performs discrete cosine transformation to generate signal X11. Adder 314 is coupled to transformation unit 312, and subtracts signal X12 from signal X11. Quantization unit 316 is coupled to adder 314, and quantifies the output of adder 314 to generate signal X13. Statistic encoding unit 318 is coupled to quantization unit 316, and statistically encodes signal X13 and motion vector MV to generate SP frame SPL. Feedback circuit 320 is coupled between quantization unit 316 and adder 314, and generates signals X12 and X14 according to signal X13. Note that transformation unit 312 can be a discrete cosine transformation unit, and statistic encoding unit 318 comprises entropy coding unit and variable length coding (VLC) unit, and motion vector MV is motion vector of P frame PL3.
Feedback circuit 320 comprises dequantization unit 322, adder 324, requantization units 326 and 330, and signal processing unit 328. Dequantization unit 322 is coupled to quantization unit 316, receives signal X13 and dequantizes the signal X13. Adder 324 is coupled to dequantization unit 322, and adds the output of dequantization unit 322 and signal X14. Requantization unit 326 is coupled between adder 324 and signal processing unit 328, and re-quantizes the output of adder 324 through quantization unit 3261 and dequantization unit 3263. Signal processing unit 328 is coupled to requantization unit 326, and generates signal X14 through inverse transformation unit 3281, filter 3282, memory device 3283, compensation unit 3284 and transformation unit 3285 according to the output of requantization unit 326. Inverse transformation unit 3281 is coupled to requantization unit 326, and dequantizes the output of requantization unit 326. Filter 3282 is coupled to inverse transformation unit 3281, filters the output of inverse transformation unit 3281 and stores the filter result to memory device 3283. Compensation unit 3284 is coupled to memory device 3283, and compensates the output of filter 3282 according to motion vector MV. Transformation unit 3285 is coupled to compensation unit 3284, and performs discrete cosine transformation on the output of compensation unit 3284 to generate signal X14. Requantization unit 330 is coupled between transformation unit 3285 and adder 314, and re-quantizes signal X14 through quantization unit 3301 and dequantization unit 3303 to generate signal X12.
Note that quantization unit 3261 and dequantization unit 3263 of requantization unit 326 and quantization unit 3301 and dequantization unit 3303 of requantization unit 330 have the same quantization parameter, and quantization unit 316 and dequantization unit 322 have the same quantization parameter. The quantization parameters of requantization unit 326 and requantization unit 330 are preferably smaller than those of quantization unit 316 and dequantization unit 322. Inverse transformation unit 3281 is an inverse discrete cosine transformation unit, and transformation unit 3285 is a discrete cosine transformation unit. Filter 3282 can be a loop filter, and compensation unit 3284 is a motion compensator.
High bitrate encoding unit 350 comprises transformation unit 352, adder 354, quantization unit 356, statistic encoding unit 358 and feedback circuit 360. Transformation unit 352 receives P frame PH3 and performs discrete cosine transformation to generate signal X21. Adder 354 is coupled to transformation unit 352, and subtracts signal X22 from signal X21. Quantization unit 356 is coupled to adder 354, and quantifies the output of adder 354 to generate signal X23. Statistic encoding unit 358 is coupled to quantization unit 356, and statistically encodes signal X23 and motion vector MV to generate SP frame SPH. Feedback circuit 360 is coupled between quantization unit 356 and adder 354, and generates signals X22 and X24 according to signal X23. Note that transformation unit 352 can be a discrete cosine transformation unit, and statistic encoding unit 358 comprises entropy coding unit and variable length coding (VLC) unit, and motion vector MV is motion vector of P frame PL3.
Feedback circuit 360 comprises dequantization unit 362, adder 364, requantization units 366 and 370, and signal processing unit 368. Dequantization unit 362 is coupled to quantization unit 356, receives signal X23 and dequantizes the signal X23. Adder 364 is coupled to dequantization unit 362, and adds the output of dequantization unit 362 and signal X25. Requantization unit 366 is coupled between adder 364 and signal processing unit 368, and re-quantizes the output of adder 364 through quantization unit 3661 and dequantization unit 3663 to generate signals X24 and X26. Signal processing unit 368 is coupled to requantization unit 366, and generates signal X25 through inverse transformation unit 3681, filter 3682, memory device 3683, compensation unit 3684 and transformation unit 3685 according to signal X26. Inverse transformation unit 3681 is coupled to requantization unit 366, and dequantizes signal X26. Filter 3682 is coupled to inverse transformation unit 3681, filters the output of inverse transformation unit 3681 and stores the filter result to memory device 3683. Compensation unit 3684 is coupled to memory device 3683, and compensates the output of filter 3682 according to motion vector MV. Transformation unit 3685 is coupled to compensation unit 3684, and performs discrete cosine transformation on the output of compensation unit 3684 to generate signal X25. Requantization unit 370 is coupled between transformation unit 3685 and adder 354, and re-quantizes signal X25 through quantization unit 3701 and dequantization unit 3703 to generate signal X22.
Note that quantization unit 3661 and dequantization unit 3663 of requantization unit 366 and quantization unit 3701 and dequantization unit 3703 of requantization unit 370 have the same quantization parameter, and quantization unit 356 and dequantization unit 362 have the same quantization parameter. The quantization parameters of requantization unit 366 and requantization unit 370 are preferably smaller than those of quantization unit 356 and dequantization unit 362. Inverse transformation unit 3681 is an inverse discrete cosine transformation unit, and transformation unit 3685 is a discrete cosine transformation unit. Filter 3682 can be a loop filter, and compensation unit 3684 is a motion compensator.
Bridge frame encoding unit 340 comprises quantization unit 342, adder 344, and statistic encoding unit 346. Quantization unit 342 quantifies signal X14. Adder 344 subtracts the output of quantization unit 342 from signal X24. Statistic encoding unit 346 statistically encodes the output of adder 344 according to motion vector MV to generate SP frame SPLH.
FIG. 2B shows a conventional server switching high bitrate bitstreams (H) to low bitrate bitstreams (L) by SP frame when the client's available bandwidth decreases. The frames received by the client are the dotted blocks PH1, PH2, SPHL, PL4, and PL5 in sequence. Note that bridge frame SPHL transmitted to client at switching point t is different with the bridge frame SPLH of FIG. 2A.
The seamless bitstream switching is achieved by the encoder 300 defined by H.264. Requantization units 326 and 330 of low bitrate encoding unit 310 and requantization units 366 and 370 of high bitrate encoding unit 350 with large quantization parameters decrease data size of the bitstream of SP frame SPLH, however, encoding efficacy of SP frame SPH and SPL is also decreased.